FTT-NAS: Discovering Fault-tolerant Convolutional Neural Architecture

With the fast evolvement of embedded deep-learning computing systems, applications powered by deep learning are moving from cloud to edge. When deploying neural networks (NNs) onto devices under complex environments, there various types possible faults: soft errors caused cosmic radiation and radioactive impurities, voltage instability, aging, temperature variations, malicious attackers, so on. Thus, safety risk NNs is now drawing much attention. In this article, after analysis faults in NN accelerators, we formalize implement fault models algorithmic perspective. We propose Fault-Tolerant Neural Architecture Search (FT-NAS) automatically discover convolutional network (CNN) architectures that reliable nowadays devices. Then, incorporate fault-tolerant training (FTT) search process achieve better results, which referred as FTT-NAS. Experiments on CIFAR-10 show discovered outperform other manually designed baseline significantly, with comparable or fewer floating-point operations (FLOPs) parameters. Specifically, same settings, F-FTT-Net feature model achieves an accuracy 86.2% (VS. 68.1% achieved MobileNet-V2), W-FTT-Net weight 69.6% 60.8% ResNet-18). By inspecting architectures, find operation primitives, quantization range, capacity model, connection pattern have influences resilience capability models.